China seeks to rein in risks from AI ‘digital humans’

BEIJING – After Ms Zhang Xinyu’s father died from cancer, she had an artificial intelligence avatar made that looked and sounded just like him, part of a growing “digital human” industry that China is moving to govern more tightly.

Videos featuring AI digital humans are ubiquitous on Chinese social media, with their uncanny features and smooth, dexterous motions often used to tout products.

The nation’s cyberspace regulator issued draft rules in April on how these avatars are developed and deployed, seeking to stop them from harming children, threatening social stability or being created to resemble someone without their consent.

Ms Zhang, 47, approached the company Super Brain two years ago, feeling depressed and lonely following her bereavement.

She can now converse online with her father’s avatar, something that made her feel “fully recharged in an instant and filled with motivation once again”, she told AFP.

Some friends worried Ms Zhang would become too immersed in the virtual world and “never be able to move on”, calling it a form of “false comfort”, she said.

“But even if the comfort itself is simulated, the love behind it is real,” added Ms Zhang, who is based in Liaoning province.

State news agency Xinhua reported in 2025 that the country’s digital human industry was worth around 4.1 billion yuan (S$764 million) in 2024, having grown by 85 per cent year on year.

Chinese governance of new digital technologies has always followed the logic of “develop first, then regulate, and perfect in the process”, said Associate Professor Marina Zhang, from the University of Technology Sydney.

‘Well-intentioned lie’

The regulations proposed by the Cyberspace Administration of China (CAC) require clear labelling on digital human content.

They also prohibit using personal information to create deepfake clones of individuals without their consent.

Super Brain founder Zhang Zewei said that he sees new laws and regulation on the sector as “inevitable”.

“I view this as a positive development, as it achieves a balance between standardised regulation and industry growth,” he told AFP.

The company specialises in creating AI avatars of the dead for grieving families.

A video clip of an elderly woman who unknowingly chatted with a hyper-realistic avatar of her dead son was widely shared on Chinese social media in April, with a related hashtag garnering more than 90 million views on Weibo.

The avatar, created by Mr Zhang’s firm, mimicked her son’s speech patterns and his movements so closely that she believed it was him on a video call.

It sparked heated online discussion about the ethics of generative AI, with some people calling for more regulation to prevent bad actors such as scammers from misusing powerful new tools.

The woman’s family approached Super Brain after her son died in a car accident, Mr Zhang told AFP.

It was a “well-intentioned lie”, he said, adding that Super Brain always obtains consent from family members of the deceased.

Guardrails

The CAC regulations – open for public comment until early May – mark China’s latest attempt to balance its technology ambitions with efforts to prevent unfettered development that could prove risky.

Violations will be punished in accordance with the law, with potential fines of 10,000 yuan to 200,000 yuan, the CAC said.

Previously, the CAC has clamped down on the use of AI-generated deepfakes that impersonate public figures in e-commerce livestreams, which it said “severely damaged” the online ecosystem.

One goal for China of imposing new tech regulations is to preserve its “sovereignty and political objectives”, said Mr Manoj Harjani, a research fellow at Singapore’s S. Rajaratnam School of International Studies.

Under the draft rules, digital humans are prohibited from generating or disseminating content that endangers national security or incites subversion of state power.

To protect children, the CAC regulations ban services offering minors virtual intimate relationships, or that encourage them to “develop extreme emotions, or cultivate harmful habits”.

“Beijing wants to move quickly on AI adoption and deployment, but within a controlled framework,” said Dr Lizzi Lee from the Asia Society Policy Institute.

There is strong support for scaling new technologies – but once “risks become visible”, Dr Lee added, regulators step in quickly. AFP